Stochastic First-Order Methods with Random Constraint Projection
نویسندگان
چکیده
We consider convex optimization problems with structures that are suitable for sequential treatment or online sampling. In particular, we focus on problems where the objective function is an expected value, and the constraint set is the intersection of a large number of simpler sets. We propose an algorithmic framework for stochastic first-order methods using random projection/proximal updates and random constraint updates, which contain as special cases several known algorithms as well as many new algorithms. To analyze the convergence of these algorithms in a unified manner, we prove a general coupled convergence theorem. It states that the convergence is obtained from an interplay between two coupled processes: progress toward feasibility and progress toward optimality. Under suitable stepsize assumptions, we show that the optimality error decreases at a rate of O(1/√k) and the feasibility error decreases at a rate of O(log k/k). We also consider a number of typical sampling processes for generating stochastic first-order information and random constraints, which are common in data-intensive applications, online learning, and simulation optimization. By using the coupled convergence theorem as a modular architecture, we are able to analyze the convergence of stochastic algorithms that use arbitrary combinations of these sampling processes.
منابع مشابه
Three Hybrid Metaheuristic Algorithms for Stochastic Flexible Flow Shop Scheduling Problem with Preventive Maintenance and Budget Constraint
Stochastic flexible flow shop scheduling problem (SFFSSP) is one the main focus of researchers due to the complexity arises from inherent uncertainties and also the difficulty of solving such NP-hard problems. Conventionally, in such problems each machine’s job process time may encounter uncertainty due to their relevant random behaviour. In order to examine such problems more realistically, fi...
متن کاملRandom Multi-Constraint Projection: Stochastic Gradient Methods for Convex Optimization with Many Constraints
Consider convex optimization problems subject to a large number of constraints. We focus on stochastic problems in which the objective takes the form of expected values and the feasible set is the intersection of a large number of convex sets. We propose a class of algorithms that perform both stochastic gradient descent and random feasibility updates simultaneously. At every iteration, the alg...
متن کاملStochastic Facilities location Model by Using Stochastic Programming
Finding the location for plans like factories or warehousesfor any organization is an important and strategic decision. Costs oftransportation which are the main part of the price of the goods, is thefunction of the location of these projects. to find the optimum locationof these projects, there have been various methods proposed which areusually defined (not random). In reality and in dealing ...
متن کاملA stabilized stochastic finite element second-order projection method for modeling natural convection in random porous media
We consider natural convection in flow saturated porous media with random porosity. The porosity is treated as a random field and a stochastic finite element method is developed. The stochastic projection method is considered for the solution of the high-dimensional stochastic Navier-Stokes equations since it leads to the uncoupling of the velocity and pressure degrees of freedom. Because of th...
متن کاملModels and formulations for multivariate dominance-constrained stochastic programs
The use of a stochastic dominance constraint to specify risk preferences in a stochastic program has been recently proposed in the literature. Such a constraint requires the random outcome resulting from one’s decision to stochastically dominate a given random comparator. These ideas have been extended to problems with multiple random outcomes, using the notion of positive linear stochastic dom...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- SIAM Journal on Optimization
دوره 26 شماره
صفحات -
تاریخ انتشار 2016